There are 2 different things here:

turning an Arduino powered device into an Linux compatible HID/Input device. This is handled by both your code on the external MCU and the host (Linux) driver. Such a driver can be implemented, as you have tried already,

with a python script, using Linux uinput support to simulate/emulate a standard Linux input device with Linux kernel module (more complicated, but perhaps better performance if you deal with GPIO).

configuring said device for emulators/EmulationStation. This part is easy as long as the first part is correctly implemented - for RetroPie it will look just like a controller. You configure the device in ES, the configuration is saved and automatically loaded each time the device in question is connected - you don't have to send any file (but see below).
Note: the auto-configuration is dependant on the device name, so if you with to have different configurations, you'd have the code developed in previous step to send a different device name for each configuration you wish to implement.

Hello,

Yes, as long as it is powered off the raspberry pi is ready to boot one 'image' or another on any drive.

The ability to boot from usb is possible and it takes some additional programming outlined here. If the usb mode is programmed the ability to boot from a SD card or a USB is granted. Additionally the ability to boot from SD card will be unchanged. Lastly the boot order while powering on will detect a installation on a SD card before a USB.

'Noobs' installation is an easy way to manage start up and it is possible to have multiple operating systems on one disk. Retropie is built on the raspbian distribution which is based on debian linux. Many of the things learned can be applied interchangeably.

Learn a lot, and you're welcome.

Thank you guys I appreciate the feedback!

@yfresh59 Great! Enjoy